BI-population CMA-ES Algorithms with Surrogate Models and Line Searches

In this paper, three extensions of the BI-population Covariance Matrix Adaptation Evolution Strategy with weighted active covariance matrix update (BIPOP-aCMA-ES) are investigated. First, to address expensive optimization, we benchmark a recently proposed extension of the self-adaptive surrogate-assisted CMA-ES which benefits from more intensive surrogate model exploitation (BIPOP-saACM-k). Second, to address separable optimization, we propose a hybrid of BIPOP-aCMA-ES and STEP algorithm with coordinatewise line search (BIPOP-aCMA-STEP). Third, we propose HCMA, a hybrid of BIPOP-saACM-k, STEP and NEWUOA to benefit both from surrogate models and line searches. All algorithms were tested on the noiseless BBOB testbed using restarts till a total number of function evaluations of 10n was reached, where n is the dimension of the function search space. The comparison shows that BIPOP-saACM-k outperforms its predecessor BIPOP-saACM up to a factor of 2 on illconditioned problems, while BIPOP-aCMA-STEP outperforms the original BIPOP-based algorithms on separable functions. The hybrid HCMA algorithm demonstrates the best overall performance compared to the best algorithms of the BBOB-2009, BBOB-2010 and BBOB-2012 when running for more than 100n function evaluations.